System and method for pick-and-drop sampling

ABSTRACT

A database system includes an input to a database server configured to deliver a data stream formed of a sequence of elements, D={p 1 , p 2 , . . . , p m } of size m of numbers from {1, . . . , n} to the database server. The system further includes a computer program that causes a processor to approximate frequency moments (F k ) in the data stream, such that a frequency of an element (i) is defined as f i =|{j:p j =i}| and a k-th frequency moment of D is defined as 
               F   k     =       ∑     i   =   1     n     ⁢       m   i   k     ⁢           ⁢   in   ⁢           ⁢   a             
single pass through the data stream. The processor is caused to carry out the steps of locating elements (i) with a frequency ΩF k  in the data stream as heavy elements and approximating f i  as ≧ a fraction of f i  to limit memory resources used by the processor to estimate F k  to O(n 1−2/k  log(n)) bits.

BACKGROUND OF THE INVENTION

The present invention relates generally to systems and methods for signal processing. More specifically, the present invention relates to a system and method for estimating frequency moments in data stream processing algorithms.

In many signal processing application, the signal must be processed in a few or, often, just one pass. For example, a “data stream” is often referred to as a sequence of data that is too large to be stored, in its entirety, in memory. Such data streams are common in communications network traffic, database transactions, satellite data feeds, and the like. In such instances, “streaming algorithms” are used to process these signals as data streams forming an input presented as a sequence of items that can be examined in very few passes. A common example of a streaming algorithm is one developed to count the number of distinct elements in a data stream. The continuous nature of the underlying signal and the resource constraints that limit the amount of repetitive processing performed general result in the algorithm producing an approximate answer based on a summary of the data stream that is stored.

Alon, Matias, and Szegedy, in The Space Complexity of Approximating the Frequency Moments, Journal of Computer and System Sciences, 58:137-147, 1999, which is incorporated herein by reference in its entirety, approached such a signal processing problem, within the context of database processing, and introduced the concept of “frequency moments.” Namely, for a sequence of elements, D={p₁, p₂, . . . , p_(m)} of size m of numbers from {1, . . . , n}, a frequency of an element, I, is defined as: f _(i) =|{j:p _(j) =i}|  Eqn. 1.

The k-th frequency moment of D is defined as:

$\begin{matrix} {F_{k} = {\sum\limits_{i = 1}^{n}{m_{i}^{k}.}}} & {{Eqn}.\mspace{14mu} 2} \end{matrix}$

Alon, Matias, and Szegedy, when approaching the problem of approximating frequency moments in one pass over D and using sublinear space, observed a striking difference between “small” and “large” values of k. Specifically, it is possible to approximate F_(k) for k≦2 in polylogarithmic space. However, polynomial space is required when k>2. Since the work of Alon, Matias, and Szegedy in the late 1990's, approximating F_(k) has become one of the most inspiring problems in the theory of data streams.

For example, many have focused on efficient algorithms for estimating particular moments, such as F₂, which is useful for computing statistical properties of the data. Others have focused on bounding the memory required of F_(k) approximation algorithms. For example, many proposed solutions or bounds having accuracy up to a polylogarithmic factor. However, as noted above, since a polynomial space is required when k>2, suitably efficient approximations for frequency moments for k≧3 have been lacking.

It would therefore be desirable to provide a system and method for approximating frequency moments with a reduced space complexity than traditional dictated for k≧3.

SUMMARY OF THE INVENTION

The present invention overcomes the aforementioned drawbacks by providing a method of non-uniform sampling to find frequency moments in a data stream.

In accordance with one aspect of the invention, a database system includes a database and a database server configured to control reading data from and writing data to the database. The system also includes an input to the database server configured to deliver a data stream formed of a sequence of elements, D={p₁, p₂, . . . , p_(m)} of size m of numbers from {1, . . . , n} to the database server. The system further includes a non-transitive, computer-readable storage medium, having stored thereon, a computer program that, when executed by a processor, causes the processor to approximate frequency moments (F_(k)) in the data stream, such that a frequency of an element (i) is defined as f_(i)=|{j:p_(j)=i}| and a k-th frequency moment of D is defined as

$F_{k} = {\sum\limits_{i = 1}^{n}m_{i}^{k}}$ in a single pass through the data stream. The processor is caused to carry out the steps of (a) arranging a portion of the data stream in a matrix, (b) selecting an initial element in the matrix, and (c) checking the matrix for a duplicate of the initial element. Upon identifying a duplicate of the initial element in the matrix, the processor is caused carry out the step of (d) assuming that the initial element appears in each row of the matrix, assigning binary values to all other frequencies, and disregarding the initial element. Upon completing step (c) without identifying a duplicate of the initial element, the processor is caused to carry out the step of (e) assigning a binary value to all frequencies. Steps (b) through (e) are repeated as step (f) for a each subsequent element in the matrix and step (g) is performed by generating a report of approximated frequency moments in the data stream

In accordance with another aspect of the invention, a method for approximating frequency moments (F_(k)) in data streams formed of a sequence of elements, D={p₁, p₂, . . . , p_(m)} of size m of numbers from {1, . . . , n} is disclosed, such that a frequency of an element (i) is defined as f_(i)=|{j:p_(j)=i}| and a k-th frequency moment of D is defined as

$F_{k} = {\sum\limits_{i = 1}^{n}{m_{i}^{k}.}}$ The method includes the steps of (a) arranging a portion of the data stream in a matrix, (b) selecting an initial element in the matrix, and (c) checking the matrix for a duplicate of the initial element. The method also includes (d) upon identifying a duplicate of the initial element in the matrix, assuming that the initial element appears in each row of the matrix, assigning binary values to all other frequencies, and disregarding the initial element. The method further includes, (e) upon completing step (c) without identifying a duplicate of the initial element, assigning a binary value to all frequencies. The method additionally includes (f) repeating steps (b) through (e) for each subsequent element in the matrix and (g) generating a report of approximated heavy elements.

In accordance with another aspect of the present invention, a a database system is disclosed that includes a database and a database server configured to control reading data from and writing data to the database. The system also includes an input to the database server configured to deliver a data stream formed of a sequence of elements, D={p₁, p₂, . . . , p_(m)} of size m of numbers from {1, . . . , n} to the database server. The system further includes a non-transitive, computer-readable storage medium, having stored thereon, a computer program that, when executed by a processor, causes the processor to approximate frequency moments (F_(k)) in the data stream, such that a frequency of an element (i) is defined as f_(i)=|{j:p_(j)=i}| and a k-th frequency moment of D is defined as

$F_{k} = {\sum\limits_{i = 1}^{n}m_{i}^{k}}$ in a single pass through the data stream. The processor is caused to carry out the steps of locating elements (i) with a frequency ΩF_(k) in the data stream as heavy elements and approximating f_(i) as ≧ a fraction of f_(i) to limit memory resources used by the processor to estimate F_(k) to O(n^(1−2/k) log(n)) bits.

The foregoing and other aspects and advantages of the invention will appear from the following description. In the description, reference is made to the accompanying drawings which form a part hereof, and in which there is shown by way of illustration a preferred embodiment of the invention. Such embodiment does not necessarily represent the full scope of the invention, however, and reference is made therefore to the claims and herein for interpreting the scope of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for use with the present invention.

FIG. 2 is a flow chart setting forth the steps of an exemplary method in accordance with the present invention and for use with systems, such as illustrated in FIG. 1.

DETAILED DESCRIPTION OF THE INVENTION

1. Introduction.

Referring to FIG. 1, a system for implementing an algorithm in accordance with the present invention is illustrated. As shown in FIG. 1, the system 10 includes a control program 12 and a memory 14 which, for example, may be resident in a database server 16 supporting database 18. Every data item to be input to the database 18 input through a data input channel 20 where the data item is processed by the control program 12.

As will be described, when a command is received from a computer system 22 accessing the system 10 the over a control channel 24 to provide an estimate of a frequency moment, the control program 12 is invoked to generate an estimated frequency moment for output on output channel 26. To this point, the present invention provides a system and method for approximating frequency moments in insertion-only data streams for k≧3. For any constant, c, the present invention can show an O(n^(1−2/k) log(n)log^((c))(n)) upper bound on the space complexity of the problem. Here log(c)(n) is the iterative log function. To simplify the presentation, the following assumptions can be made:

n and m are polynomially far; and

approximation error ε and parameter k are constants.

A natural bijection between streams and special matrices can be observed. As described hereafter, the present invention provides a non-uniform sampling method referred to herein as “a pick-and-drop sampling.”

To illustrate a pick-and-drop method in accordance with the present invention, an example can be utilized where m=r*t and r=[n^(1/5)]. In this context and referring to FIG. 2, the following example of a sampling method 200 in accordance with the present invention is described. At process block 202, the data stream is arranged into a matrix. Arranging the data stream in the matrix may be achieved via a logical arrangement. By way of the present non-limiting example, consider r×t matrix, M, with entries m_(ij)=p_(k(i−1)+j). For m≦n, the following promise problem can be solved with probability ⅔:

case 1—all frequencies are either zero or one;

case 2—z appears in every row of M exactly once and, thus, f_(z)=r; and

all other frequencies are either zero or one.

At process block 204, an element is selected for analysis. The element may be selected at random. Specifically, r independent and identically distributed (i.i.d.) random numbers I₁, . . . , I_(r), can be picked, where I_(i) is uniformly distributed on {1,2, . . . , t}. At decision block 206, for a given element (and, as will be described, each i=1 . . . r−1), a check for a duplicate of m_(i,I) _(i) in the row i+1 is made. If the duplicate is found, in this example, then we output “case 2” by assuming that z appears in every row of M exactly once. More generally, at process block 208, the i-th sample is “dropped,” and, at process block 210, another element is selected, for example, to “pick” the (i+1)-th sample. This process is repeated T times independently.

Returning to decision block 206, if no duplicate is found, at decision block 212, a check is made to determine whether the end of the matrix has been reached. If not, another element is selected at process block 210. Once the end of the matrix is reached, at process block 212, a report is generated, for example, to report the “heavy elements.” Heavy elements are the element that appear often in the data stream. Frequency moments area a function of the data stream. As will be described, the present invention provides a system and method for determining a frequency moment using heavy elements, but the heavy element can be used for other and additional purposes.

By way of the present example, “case 1” is output in the report if no duplicate is found. Note, if the input represents case 1, the method will always output “case 1.” Consider case 2 and observe that, if m_(i,I) _(i) =z, then case 2 will be output. Indeed, since z appears in every row, the duplicate of z will be found. The probability to miss z entirely is:

$\begin{matrix} {\left( {1 - \frac{1}{t}} \right)^{r\; T}.} & {{Eqn}.\mspace{14mu} 3} \end{matrix}$

Recall that m≦n,m=rt,r=[n^(1/k)]. If T=O(n^(1−2/k)) with sufficiently large constant, then the probability of error with respect to eqn. 3 is smaller than ⅓. Accordingly, the promise problem can be resolved with O(n^(1−2/k) log(n)) space. Notably, the solution depends upon r. Thus, in general, it is prudent to carefully select the matrix.

Unfortunately, the distribution of the frequent element in the stream can be arbitrary. Also the algorithm should desirably recognize “noisy” frequencies that are large but negligible. Clearly, the sampling would be desirable if more intricate but, luckily, it need not be rendered greatly more complex.

Accordingly, still referring to FIG. 2, counters can be used. Specifically, a local counter can be introduced for each sample that counts the number of times m_(i,I) _(i) appears in the suffix of the i-th row. Notably, such a counting method was used by Alon, Matias, and Szegedy, in The Space Complexity of Approximating the Frequency Moments, for the entire stream. In contrast, in the present invention it is contemplated that a global sample (and a global counter), as functions of the local samples and counters, may be used. Initially, at process block 216 the global sample is the local sample of the first row and incremented at process block 218 only when the local counter indicates such.

Notably, under certain conditions, the global sample can be “dropped.” If this is the case, then the local sample of the current row is “picked” and becomes the new global sample. The global sample is “dropped” when the local counter exceeds the global one. Also, the global sample is dropped if the global counter does not grow fast enough. A function, λq, where λ is a parameter and q is the number of rows that the global counter survived. If the global counter is smaller than λq, then the global sample is “dropped.”

2. Pick-and-Drop Sampling

Let M be a matrix with r rows and t columns and with entries m_(i,j)ε[n]. For iε[r], jε[t], lε[n] define:

$\begin{matrix} {{d_{i,j} = {\left\{ {{{j^{\prime}\text{:}j} \leq j^{\prime} \leq t},{m_{i,j^{\prime}} = m_{i,j}}} \right\} }};} & {{Eqn}.\mspace{14mu} 4} \\ {{f_{l,i} = {\left\{ {{j \in {\left\{ t \right\}\text{:}m_{i,j}}} = l} \right\} }};} & {{Eqn}.\mspace{14mu} 5} \\ {{f_{l} = {\left\{ {{\left( {i,j} \right)\text{:}m_{i,j}} = l} \right\} }};} & {{Eqn}.\mspace{14mu} 6} \\ {{F_{k} = {\sum\limits_{l = 1}^{n}f_{l}^{k}}},{G_{k} = {F_{k} - {f_{1}^{k}.}}}} & {{Eqn}.\mspace{14mu} 7} \end{matrix}$

Note that there is a bijection between r×t matrices M and streams D of size r×t with elements p_(it+j)=m_(i,j), where the definitions with respect to eqns. 2, 1 and 6, 7, define equivalent frequency vectors for a matrix and the corresponding stream. Without loss of generality, consider streams of size r×t for some r,t. The notions of a stream and its corresponding matrix can be interchanged.

Let {I_(j)}_(j=1) ^(r) be i.i.d. random variables with uniform distributions on [t]. Define for i=1, . . . , r: s_(i)=m_(i,I) _(i) , c_(i)=d_(i,I) _(i)   Eqn. 8.

Let λ be a parameter. Define the following recurrent random variables: S₁=s₁, C₁=c, q₁=1  Eqn. 9.

Also (for i=2, . . . r), if: (C_(i−1)<max{λq_(i−1),c_(i)})  Eqn. 10;

then define: S_(i)=s_(i), C_(i)=c_(i), q_(i)=1  Eqn. 11;

otherwise, define: S _(i) =S _(i−1) , C _(i) =C _(i−1) +f _(S) _(i) _(,i) , q _(i) =q _(i−1)+1  Eqn. 12.

Therefore, Theorem 2.1 states: If M is a r×t matrix, there exist absolute constants α,β such that, if:

$\begin{matrix} {{{\alpha\left( {{\lambda\; r} + \frac{G_{3}}{\lambda\; t} + \frac{G_{2}}{t}} \right)} \leq f_{1} \leq {\beta\; t}};} & {{Eqn}.\mspace{14mu} 13} \end{matrix}$

then:

$\begin{matrix} {{P\left( {S_{r} = 1} \right)} \geq {\frac{f_{1}}{2\; t}.}} & {{Eqn}.\mspace{14mu} 14} \end{matrix}$

Proof. Denote Q={(i,j):m_(i,j)=1}. For (i,j)εQ, define: T _(i,j)= (A _(i,j) ∪B _(i,j) ∪H _(i,j))  Eqn. 15;

where for i>1: A _(i,j)=((C _(i−1) ≧d _(i,j))∩(S _(i−1)≠1))  Eqn. 16;

for i<r:

$\begin{matrix} {{B_{i,j}\left( {\underset{h = {i + 1}}{\bigcup\limits^{r}}\left( {{d_{i,j} + {\sum\limits_{u = {i + 1}}^{h - 1}f_{1,u}}} < c_{h}} \right)} \right)};} & {{Eqn}.\mspace{14mu} 17} \\ {{H_{i,j}\left( {\overset{r}{\bigcup\limits_{h = {i + 1}}}\left( {{d_{i,j} + {\sum\limits_{u = {i + 1}}^{h - 1}f_{1,u}}} < {\left( {h - 1} \right)\lambda}} \right)} \right)};} & {{Eqn}.\mspace{14mu} 18} \end{matrix}$

and A_(i,j)=B_(r,j)=H_(r,j)=0. We have: ((s _(i)=1)∩(S _(i−1)≠1)∩ A_(i,I) _(i) ⊂((s _(i)=1)∩(C _(i−1) >c _(i))) ⊂((S _(i)=1)∩(q _(i)=1))  Eqn. 19

Consider the case when S_(i)=1 and q_(i)=1 and

${d_{i,I_{i}} + {\sum\limits_{u = {i + 1}}^{h - 1}f_{1,u}}} \geq {{\max\left( {{\lambda\left( {h - i} \right)},c_{h}} \right)}\mspace{14mu}{for}\mspace{14mu}{all}\mspace{14mu} h} > {i.}$ In this case S_(h) will be defined by eqn. 12 and not by eqn. 11. In particular, S_(h)=S_(i)=1. Therefore:

$\begin{matrix} \left( {{\left( {S_{i} = 1} \right)\bigcap\left( {q_{i} = 1} \right)\bigcap\overset{\_}{B_{i,I_{i}}}\bigcap\overset{\_}{H_{i,I_{i}}}} \subseteq {\left( {\overset{r}{\bigcap\limits_{h = i}}\left( {S_{h} = 1} \right)} \right).}} \right. & {{Eqn}.\mspace{14mu} 20} \end{matrix}$

Define V₁=((s₁=1)∩T_(i,I) ₁ ) and, for i>1, V_(i)=((s_(i)=1)∩(S_(i−1)≠1)∩T_(i,I) _(i) ) . It follows from eqns. 19 and 20 that, for any iε[r]: V _(i) ⊂(S _(r)=1)  Eqn. 21; V_(i)∩V_(j)=0  Eqn. 22.

Thus:

$\begin{matrix} {{\sum\limits_{i = 1}^{r}{P\left( {V_{i}\overset{r}{\bigcup\limits_{i = 1}}V_{i}} \right)}} \leq {{P\left( {S_{r} = 1} \right)}.}} & {{Eqn}.\mspace{14mu} 23} \end{matrix}$

For any i>1, P(V_(i))≧P((s_(i)=1)∩T_(i,I) _(i) )−P(s_(i)=S_(i−1)=1). Also,

$\left( {{{\sum\limits_{i = 2}^{r}{{P\left( {s_{i} = {S_{i - 1} = 1}} \right)}{\sum\limits_{i = 2}^{r}{P\left( {\left( {s_{i} = I} \right)\bigcap\left( {\bigcup_{h \neq i}\left( {s_{h} = 1} \right)} \right)} \right)}}}} \leq \left( {\sum\limits_{i = 1}^{r}{P\left( {s_{i} = 1} \right)}} \right)^{2}} = {\left( \frac{f_{1}}{t} \right)^{2}.}} \right.$

For any fixed (i,j)εQ events I_(i)=j and T_(i,j) are independent. Indeed, A_(i,j) is defined by {S_(i−1), C_(i−1)} that, in turn, id defined by {I₁, . . . , I_(i−1)}. Similarly, B_(i,j) is defined by {I_(i+1), . . . , I_(r)}. Note that H_(i,j) is a deterministic event. By definition, {I₁, . . . , I_(i−1), I_(i+1), . . . , I_(r)} are independent of I_(i). Thus, event I_(i)=j and T_(i,j)= (A_(i,j)∪B_(i,j)∪H_(i,j)) are independent. Thus:

$\begin{matrix} \begin{matrix} {{\sum\limits_{i = 2}^{r}{P\left( {\left( {s_{i} = 1} \right)\bigcap T_{i,I_{i}}} \right)}} = {\sum\limits_{{({i,j})} \in Q}{P\left( {\left( {I_{i} = j} \right)\bigcap T_{i,j}} \right.}}} \\ {= {\sum\limits_{{({i,j})} \in Q}{{P\left( {I_{i} = j} \right)}{P\left( T_{i,j} \right)}}}} \\ {= {\frac{1}{t}{\sum\limits_{{({i,j})} \in Q}{{P\left( T_{i,j} \right)}.}}}} \end{matrix} & {{Eqn}.\mspace{14mu} 24} \end{matrix}$

Thus,

${P\left( {S = 1} \right)} \geq {{\frac{1}{t}{\sum\limits_{{({i,j})} \in Q}{P\left( T_{i,j} \right)}}} - {\left( \frac{f_{1}}{t} \right)^{2}.}}$ As will be clear, Lemma 2.2 implies that Σ_((i,j)εQ)P(T_(i,j))≧0.8f₁. Thus, if β<0.3, then

${{P\left( {S_{r} = 1} \right)}\frac{f_{1}}{t}\left( {0.8 - \frac{f_{1}}{t}} \right)} \geq {\frac{f_{1}}{2t}.}$ Here, only the second part of eqn. 13 was used. The first part is used in the proof of Lemma 2.2.

Lemma 2.2. There exists absolute constants α,β such that eqn. 13 implies

${\sum\limits_{{({i,j})} \in Q}{P\left( T_{i,j} \right)}} > {0.8\mspace{14mu}{f_{1}.}}$

It follows from Lemmas 2.9, 2.17, and 2.14 and the union bound that there exists at lest 0.97f₁ pairs (i,j)εQ, such that P(A_(i,j)∪B_(i,j)∪H_(i,j))≦0.02. Recall that T_(i,j)= (A_(i,j)∪B_(i,j)∪H_(i,j)). The lemma follows.

Events of type A.

For (i,j)εQ_(such that) i>1 and for l>1, define:

${Y_{l,{({i,j})}} = {1_{A_{i,j}}1_{({S_{i - 1} = l})}}},{Y_{l,i} = {\sum\limits_{{j \in {\lbrack t\rbrack}},{{({i,j})} \in Q}}Y_{l,{({i,j})}}}},{Y_{l} = {\sum\limits_{i = 2}^{r}Y_{l,i}}},{Y = {\sum\limits_{l = 2}^{n}Y_{l}}},$

Fact 2.3. C_(i)≦fs_(i),i. Also, if q_(i)=1, then C_(i)≦fs_(i),i.

Proof. It follows directly from eqns. 11 and 12 that it is sufficient to prove, for any i, there exists a set Q_(i), such that C_(i)≦|Q_(i)| and, simultaneously, Q_(i) is a subset of {(i′,j):m_(i′,j)=S_(i), i′≦i}. Through the induction on i, the above claim can be proven. For i=1, the claim is true since we can define Q₁={(1,j):j≧I₁}. For i>2, the description of the algorithm implies the following. If q_(i)=1, then we can put Q_(i)={(i,j):j≧I_(i)}. If q_(i)>1, then define Q_(i)=Q_(i−1)∩{(i,j):m_(i,j)=S_(i)}. Note that in this case, S_(i)=S_(i−1). The second part follows form the description of the algorithms. Namely, if p_(i)=1, then C_(i)=c_(i), S_(i)=s_(i), and c_(i)=d_(i,I) _(i) (s)≦f_(s) _(i) _(,i).

Fact 2.4.

1. Y_(l,i)≦f_(l),

2. If q_(i−1)=1, then Y_(l,i)≦f_(l,i−1)

Proof. Let (i,j)εQ be such that d_(i,j)>f_(l); then Y_(l,(i,j))=1_((c) _(i−1) _(≧d) _(i,j) ₎1_((s) _(i−1) ₌₁₎=1_((f) _(l) _(≧c) _(i−1) ₎1_((c) _(i−1) _(≧d) _(i,j)) 1_((s) _(i−1) ₌₁₎. Use Fact 2.3 for this last equality. Thus, Y_(l,(i,j))=0. The definition of d_(i,j) implies |{j:(i,j)εQ,d_(i,j)≦f_(l)f}|≦f_(l) for any fixed i and l. Thus,

$Y_{l,i} = {{\sum\limits_{{j \in {\lbrack t\rbrack}},{{({i,j})} \in Q}}Y_{l,{({i,j})}}} \leq {f_{l}.}}$ Part 2 follows by repeating the above arguments and using the second statement of Fact 2.3.

Definition 2.5. Let 1≦r₁≦r₂≦r and lε[n]. Call a pair [r₁,r₂] an I-epoch if ∀i=r₁, . . . , r₂: S_(i)=l and q_(r1)=q_(r2+1)=1, and ∀i=r+1₁, . . . , r₂:q_(i)=q_(i−1)+1.

Lemma 2.6. Let [r₁,r₂] be an I-epoch. If r₂>r₁, then,

${r_{2} - r_{1}} \leq {\frac{1}{\lambda}{\sum\limits_{i = r_{1}}^{r_{2} - 1}{f_{l,i}.}}}$

Proof. First, observe that q_(r) ₂ ⁻¹=r₂−r₁. Second, q_(i)>1 implies that S_(i) is defined by eqn. 12 and not by eqn. 11 for all r₁<i≦r₂. In particular, C_(r1)≦f_(l,r) ₁ and for r₁<i≦r₂ we have C_(i)=C_(i−1)+f_(l,i). Thus,

$C_{r_{2} - 1} \leq {\sum\limits_{i = r_{1}}^{r_{2} - 1}{f_{l,i}.}}$ Third, C_(r) ₂ ⁻¹≧λq_(r) ₂ ⁻¹, since eqn. 10 must be false for i=r₂. Therefore,

${r_{2} - r_{1}} = {{q_{r_{2}} - 1} \leq {\frac{1}{\lambda}C_{r_{2} - 1}} \leq {\frac{1}{\lambda}{\sum\limits_{i = r_{1}}^{r_{2} - 1}{f_{l,i}.}}}}$

Lemma 2.7.

$Y_{l} \leq {\frac{f_{l}^{2}}{\lambda} + {f_{l}.}}$

Proof. Observe that the set {i:S_(i)=l} is a collection of disjoint I-epochs. Recall that and Y_(l)Σ_(i=2) ^(r) Y_(l,i) and Y_(l,i) is non-zero only if S_(i−1) is equal to l. Thus, Y_(l) can be rewritten as

$Y_{l} = {\sum\limits_{{{({r_{1},r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}{\left( {\sum\limits_{i = {r_{1} + 1}}^{r_{2} + 1}Y_{l,i}} \right).}}$ For any epoch such that r₂>r₁, we have by Lemmas 2.4 and 2.6:

$Y_{l} = {{{\sum\limits_{{{({r_{1} < r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}\left( {\sum\limits_{i = {r_{1} + 1}}^{r_{2} + 1}Y_{l,i}} \right)} + {\sum\limits_{{{({r_{1} = r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}Y_{l,{r_{2} + 1}}}} = {{{\sum\limits_{{{({r_{1} < r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}\left( {\sum\limits_{i = {r_{1} + 1}}^{r_{2}}Y_{l,i}} \right)} + {\sum\limits_{{{({r_{1},r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}Y_{l,{r_{2} + 1}}}} \leq {{\frac{f_{l}}{\lambda}{\sum\limits_{{{({r_{1} < r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}\left( {\sum\limits_{i = r_{1}}^{r_{2} - 1}f_{l,i}} \right)}} + {\sum\limits_{{{({r_{1},r_{2}})}{is\_ an}{\_ l}} - {epoch}}^{\;}f_{l,{r_{2} + 1}}}} \leq {\frac{f_{l}^{2}}{\lambda} + {f_{l}.}}}}$

Lemma 2.8.

${P\left( {Y_{l} > 0} \right)} \leq {\frac{f_{l}}{t}.}$

Proof. Since I_(i) are independent and

$0 \leq \frac{f_{l,i}}{t} \leq 1$ we can apply Fact 2.10 as

${P\left( {\bigcap_{i = 1}^{r}\left( {m_{i,I_{i}} \neq l} \right)} \right)} = {{\prod\limits_{i = 1}^{r}\;\left( {1 - \frac{f_{l,i}}{t}} \right)} \geq {\left( {1 - \frac{f_{l}}{t}} \right).}}$ Thus:

$\begin{matrix} {{{P\left( {Y_{l} > 0} \right)}{P\left( {\bigcup_{i = 1}^{r}\left( {m_{i,I_{i}} = l} \right)} \right)}} \leq {\frac{f_{l}}{t}.}} & {{Eqn}.\mspace{14mu} 25} \end{matrix}$

Lemma 2.9. There exists an absolute constant α such that eqn. 13 implies that P(A_(i,j))≦0.01 for at least 0.99 f₁ pairs (i,j)εQ.

Proof. From Lemmas 2.7 and 2.8,

${{E\left( Y_{l} \right)} \leq {\frac{f_{l}}{t}\left( {\frac{f_{l}^{2}}{\lambda} + f_{l}} \right)}},{{E(l)} = {{\sum\limits_{l = 2}^{n}{E\left( Y_{l} \right)}} \leq {\frac{G_{3}}{\lambda\; t} + {\frac{G_{2}}{t}.}}}}$ It follows that Σ_((i,j)εQ)1_(A) _(i,j) −Y. Recall that by eqn. 13,

${Q} = {f_{1} \geq {\alpha\left( {\frac{G_{3}}{\lambda\; t} + \frac{G_{2}}{t}} \right)} \geq {\alpha\;{{E\left( {\sum\limits_{{({i,j})} \in Q}^{\;}1_{{Ai},j}} \right)}.}}}$ Fact 2.11 implies that there exists an absolute constant α such that the lemma is true.

The following fact is well known. For completeness, the proof is presented below.

Fact 2.10. Let α₁, . . . , α_(r) be real numbers in [0,1]. Then,

${\prod\limits_{i = {{1i} = 1}}^{r}\;\left( {1 - \alpha_{i}} \right)} \geq {1 - {\left( {\sum\limits^{r}\alpha_{i}} \right).}}$

Proof. If Σ_(i=1) ^(r)α_(i)≧1, then

${\prod\limits_{i = {{1i} = 1}}^{r}\;\left( {1 - \alpha_{i}} \right)} \geq 0 \geq {1 - {\left( {\sum\limits^{r}\alpha_{i}} \right).}}$

Thus, we can assume that Σ_(i=1) ^(r)α_(i)<1. This claim can be proven by induction on r. For r=2, we obtain (1−α₁)(1−α₂)=(1−α₁−α₂x+α₁α₂)≧(1−α₁−α₂). For r>2, we have, by induction,

${\prod\limits_{i = 1}^{r}\;\left( {1 - \alpha_{i}} \right)} \geq {\left( {1 - \left( {\sum\limits_{i = 1}^{r - 1}\alpha_{i}} \right)} \right)\left( {1 - \alpha_{r}} \right)} \geq {1 - {\left( {\overset{r}{\sum\limits_{i = 1}}\alpha_{i}} \right).}}$

Fact 2.11. Let X₁, . . . , X_(u) be a sequence of indicator random variables. Let S={i:P(X_(l)=1)≦v}. If E(Σ_(i=1) ^(u)X_(i))≦μu then

${S} \geq {\left( {1 - \frac{\mu}{v}} \right){u.}}$

Proof. Indeed,

${\mu\; u} \geq {\sum\limits_{i \notin S}^{\;}{P\left( {X_{i} = 1} \right)}} \geq {{v\left( {u - {S}} \right)}.}$

Events of type B.

For (i,j)εQ let Z_((i,j))=1_(B) _(i,j) . Let Z=Σ_((i,j)εQ)Z_((i,j)). We use arguments that are similar to the ones from the previous section. To stress the similarity, we abuse the notation and denotation by Y_(l,j(i,j)) the indicator of the event that h>i+1,s_(h)=l and

$\left( {d_{i,j} + {\sum\limits_{u = {i + 1}}^{h - 1}f_{1,u}}} \right) < {c_{h}.}$

Define Y_(l,j)=Σ_((i,j)εQ)Y_(l,h,(i,j)), Y_(l)=Σ_(h=1) ^(r)Y_(l,h).

Fact 2.12. Y_(l)≦f_(l).

Proof. Repeating the arguments from Fact 2.4, we have c_(h)1_(s) _(h) ₌₁≦f_(l,h) and thus Y_(l,h)≦f_(l,h).

Fact 2.13.

${P\left( {Y_{l} > 0} \right)} \leq {\frac{f_{l}}{t}.}$

Proof. The proof is identical to the proof of Lemma 2.8.

Lemma 2.14. There exists absolute constants α,β such that eqn. 13 implies that P(B_(i,j))≦0.01 for at least 0.99 f₁ pairs (i,j)εQ.

Proof. Denote Y=Σ_(l=1) ^(n) Y_(l). It follows that Z≦Y and E(Z)≦E(Y). By Facts 2.13 and 2.12, it follows that

${E\left( Y_{l} \right)} \leq {\frac{f_{l}^{2}}{t}.}$ Thus, by eqn. 13,

${{E(Z)} \leq {E(Y)} \leq \frac{F_{2}}{t}} = {{\frac{G_{2}}{t} + {f_{1}\frac{f_{1}}{t}}} \leq {\left( {\alpha + \beta} \right){f_{1}.}}}$

We repeat the arguments from Lemma 2.9.

Events of type H

Definition 2.15.

Let U={u₁, . . . , u_(t)} and W={w₁, . . . , w_(t)} be two sequences of non-negative integers. Let (i,j) be a pair such that 1≦i≦t and 1≦j≦u_(i). Denote (i,j) as a loosing pair (with respect to sequences U,W) if there exists h,i≦h≦t such that

${{- j} + {\sum\limits_{s = i}^{h}\left( {u_{s} - w_{s}} \right)}} < 0.$ Denote any pair that is not a loosing pair as a winning pair.

Now, we consider the following pair (U,W) then H_(i,j′) does not occur where j′ is such that m_(i,j′=1) and d_(i,j′)=f_(1,i)−j+1.

Proof. By Definition 2.15, for every i≦h≦r:

$\begin{matrix} {{{- j} + {\sum\limits_{l = i}^{h}u_{l}}} \geq {\sum\limits_{l = i}^{h}{w_{l}.}}} & {{Eqn}.\mspace{14mu} 26} \end{matrix}$

Since Σ_(l=i) ^(h)w_(i)=(h−i+1)λ and d_(i,j′)=f_(1,i)−j+1, for every i≦h≦t,

${d_{i,j^{\prime}} + {\sum\limits_{l = {i + 1}}^{h}d_{l,1}}} = {{{- j} + 1 + {\sum\limits_{l = {i + 1}}^{h}f_{l,1}}} = {{{{- j} + 1 + {\sum\limits_{l = i}^{h}u_{l}}} \geq {{- j} + {\sum\limits_{l = i}^{h}u_{l}}} \geq {\sum\limits_{l = i}^{h}w_{l}}} = {\left( {h - i + 1} \right)\lambda}}}$

Substitute h by h−1 (for h>i), such that

${d_{i,j^{\prime}} + {\sum\limits_{l = {i + 1}}^{h}d_{l,1}}} \geq {\left( {h - i} \right){\lambda.}}$ Thus, H_(i,j′) does not occur, by eqn. 18.

Lemma 2.17. There exists an absolute constant α, such that eqn. 13 implies that H_(i,j) does not occur for at least 0.99 f₁ pairs (i,j)εW.

Proof. By Lemma 2.20, there exists at least

$\sum\limits_{i = 1}^{r}\left( {u_{i} - w_{i}} \right)$ winning pairs (i,j) with respect to the (U,W). Also, Σ_(i=1) ^(r)u_(i)=Σ_(i=1) ^(r)f_(1,i)=f₁ and Σ_(i=1) ^(r)uw=λr. Thus, there exist at least f₁−λr winning pairs (i,j) with respect to the (U,W). In the statement of Fact 2.16, the mapping from j to j′ is bijection. Thus, there exist at exist f₁−λr winning pairs (i,j′) such that m_(i,j′)=1 and H_(i,j′) does not occur. By eqn. 13, we have f₁≧αλr and the lemma follows.

Definition 2.18. Let U={u₁, . . . , u_(t)} and W={w₁, . . . , w_(t)} be two sequences of non-negative integers. Let 1≦h<t. Let U′,W′ be two sequences of size t−h defined by p′_(i)=u_(i+h),q′_(i)=w_(i+h) for i=1, . . . , t−h. Denote U′,W′ as h-tail of the sequences U,W.

Fact 2.19. If (i,j) is a winning pair with respect to h-tail of U,W then (i+h,j) is a winning pair with respect to U,W. If (i,j) is a winning pair with respect to h-tail of U,W the (i,j) is a winning pair with respect to U,W.

Proof. Follows directly from Definitions 2.15 and 2.18.

Lemma 2.20. If Σ_(s=1) ^(t)(u_(s)−w_(s))>0, then there exist at least Σ_(s=1) ^(t)(u_(s)−w_(s)) winning pairs.

Proof. We use induction on t. For t=1, any pair (1,j) is winning if 1≦j≦u₁−w₁. Consider t>1 and apply the following case analysis.

1. Assume that there exist 1≦h<t, such that Σ_(s=1) ^(h)(u_(s)−w_(s))≦0. Consider the h-tail of U,W. By induction and by Fact 2.19, there exist at least Σ_(s=h+1) ^(t)(u_(s)−w_(s))≧Σ_(s=1) ^(t)(u_(s)−w_(s)) winning pairs with respect to U,W.

2. Assume that (1,u₁) is a winning pair. It follows that (1,j), j<u₁ is a winning pair as well. If Σ_(s=2) ^(t)(u_(s)−w_(s))>0, then, by induction and by Fact 2.19, there exist at least Σ_(s=2) ^(t)(u_(s)−w_(s)) winning pairs of the form (i,j) where i>1. In total there are u₁+Σ_(s=2) ^(t)(u_(s)−w_(s))≧Σ_(s=1) ^(t)(u_(s)−w_(s)) winning pairs with respect to U,W. The case when Σ_(s=2) ^(t)(u_(s)−w_(s))<0 is trivial.

Assume that eqns. 1 and 2 do not hold. Then, u₁>0. Indeed otherwise u₁−w₁≦0 and, thus, eqn. 1 is true. Also, (1,1) is a winning pair. Indeed, otherwise there exists 1<h<t such that −1+Σ_(i=1) ^(t)(u_(i)−w_(i))<0. All numbers are integers. Thus, Σ_(i=1) ^(h)(u_(i)−w_(i))≦0 and eqn. 1 is true. Thus, (1,1) is a winning pair and (1,u₁) is not a winning pair (by eqn. 2). Therefore, there exist 1<u≦u₁, such that (1,u−1) is a winning pair and (1,u) is not a winning pair. In particular, there exist 1≦h<t, such that

${{- u} + {\sum\limits_{s = 1}^{h}\left( {u_{s} - w_{s}} \right)}} < 0.$ On the other hand, (1,u−1) is a winning pair, thus,

$0 \leq {1 - u + {\sum\limits_{s = 1}^{h}{\left( {u_{s} - w_{s}} \right).}}}$ All numbers are integers and, thus, it can be concluded that

${\sum\limits_{s = 1}^{h}\left( {u_{s} - w_{s}} \right)} = {u - 1.}$ Consider the h-tail of U,W. By induction, there exists at least

${\sum\limits_{i = {h + 1}}^{t}\left( {u_{i} - w_{i}} \right)} = {{\sum\limits_{i = 1}^{t}\left( {u_{i} - w_{i}} \right)} - \left( {u - 1} \right)}$ winning pairs with respect to the h-tail of U,W of the form (i,j) , where i>1. By properties of u, there exist additional (u−1) winning pairs of the form (1,j), j≦u−1. Summing up, the fact is obtained.

3. The Streaming Algorithm

Fact 3.1. Let v₁, . . . , v_(n) be a sequence of non-negative numbers and let k>2. Then,

$\left( {\sum\limits_{i = 1}^{n}v_{i}^{2}} \right)^{({k - 1})} \leq {\left( {\sum\limits_{i = 1}^{n}v_{i}^{k}} \right){\left( {\sum\limits_{i = 1}^{n}v_{i}} \right)^{({k - 2})}.}}$

Proof. Define

$\lambda_{i} = {\frac{v_{i}}{\sum\limits_{j = 1}^{n}v_{j}}.}$ Since g(x)=x^(k−1) is convex on the interval [0,∞), we can apply Jensen's inequality and obtain

$\left( \frac{\sum\limits_{i = 1}^{n}v_{i}^{2}}{\sum\limits_{i = 1}^{n}v_{i}} \right)^{k - 1} = {{\left( {\sum\limits_{i = 1}^{n}{\lambda_{i}v_{i}}} \right)^{({k - 1})} \leq \left( {\sum\limits_{i = 1}^{n}{\lambda_{i}v_{i}^{({k - 1})}}} \right)} = {\frac{\sum\limits_{i = 1}^{n}v_{i}^{k}}{\sum\limits_{i = 1}^{n}v_{i}}.}}$

Let D be a stream. Define:

$\begin{matrix} {{\psi = \frac{n^{1 - {({1/k})}}G_{k}^{1/k}}{F_{1}}},{\delta = {2\left\lceil {0.5\;{\log_{2}(\psi)}} \right\rceil}},{t = \left\lceil \frac{\delta\; F_{1}}{n^{1/k}} \right\rceil},{{\lambda = \left\lceil \frac{F_{1}\delta^{3}}{n} \right\rceil};}} & {{Eqn}.\mspace{14mu} 27} \end{matrix}$

where eqn. 2 was used to define F_(k). We will make the following assumptions: f ₁≦0.1F ₁ , t≦F ₁ ,F ₁(modt)=0  Eqn. 28.

Then, it is possible to define a r×t matrix M, where r=F₁/t and with entries m_(i,j)=p_(ir+j).

Fact 3.2. 1≦δ≦2n^((k−1)/2k).

Proof. Indeed G₁≦G_(k) ^(1/k)n^(1−1/k) by Holder inequality and since f₁≦0.1F₁ by eqn. 28, we have ψ≧0.5. Thus, ┌0.5 log₂(ψ)┐≧0 and the lower bound follows. Also, F_(k) ^(1/k) is the L_(k) norm for the frequency vector since all frequencies are non-negative. Since L_(k)≦L₁, we conclude that ψ≦n^(1−1/k) and the fact follows.

Observe that there exists a frequency vector with δ=O(1): put f_(j)=1 for all iε[n]. At the same time there exists a vector with δ=Ω(n^((k−1)/2k)): put f₁=n and f_(j)=1 for j>2. It is not hard to see that if δ is sufficiently large ten a naive sampling method will find a heavy element. For example, in the latter case, the heavy element occupies half of the stream.

Fact 3.3. λr≦4G_(k) ^(1/k).

Proof. Recall that F₁=rt. The fact follows from the definitions of λ and t.

Fact 3.4.

$\frac{G_{2}}{t} \leq {G_{k}^{1/k}.}$

Proof. Define

$\alpha = {\frac{k - 3}{2\left( {k - 2} \right)}.}$ We have by Holder inequality:

$\begin{matrix} {{G_{2}^{\alpha} \leq {G_{k}^{\frac{2\alpha}{k}}n^{\alpha{({1 - \frac{2}{k}})}}}} = {G_{k}^{\frac{k - 3}{k{({k - 2})}}}{n^{\frac{k - 3}{2k}}.}}} & {{Eqn}.\mspace{14mu} 29} \end{matrix}$

Also, by Fact 3.1:

$\begin{matrix} {G_{2}^{1 - \alpha} = {G_{k}^{\frac{k - 1}{2{({k - 2})}}} \leq {G_{k}^{\frac{1}{k{({k - 2})}}}{G_{1}^{\frac{1}{2}}.}}}} & {{Eqn}.\mspace{14mu} 30} \end{matrix}$

Thus,

${G_{2} \leq {G_{k}^{\frac{k - 3}{k{({k - 2})}}}n^{\frac{k - 3}{2k}}G_{k}^{\frac{1}{2{({k - 2})}}}F_{1}^{\frac{1}{2}}}} = {{G_{k}^{\frac{1}{k}}\frac{F_{1}}{n^{1/5}}\left( \frac{G_{k}^{\frac{1}{k}}n^{\frac{k - 1}{k}}}{F_{1}} \right)^{1/2}} = {t\;{G_{k}^{\frac{1}{k}}.}}}$

Fact 3.5.

$\frac{G_{3}}{\lambda\; t} \leq {G_{k}^{1/k}.}$

Proof. By Holder inequality: G ₃≦G_(k) ^(3/k) n ^(1−(3/k))  Eqn. 31.

Thus,

$\frac{G_{3}}{\lambda\; t} = {\frac{n^{1 + {({1/k})}}G_{3}}{F_{1}^{2}\delta^{4}} \leq \frac{n^{2 - {({2/k})}}G_{k}^{3/k}}{F_{1}^{2}\delta^{4}} \leq {G_{k}^{1/k}.}}$

Theorem 3.6. Let M be a r×t matrix, such that eqn. 27 is true. Then, there exist absolute constants α,β such that: αG _(k) ^(1/k) ≦f ₁ ≦βt  Eqn. 32;

imply:

$\begin{matrix} {{P\left( {S_{r} = 1} \right)} \geq {\frac{\delta}{2\; n^{1 - {({2/k})}}}.}} & {{Eqn}.\mspace{14mu} 33} \end{matrix}$

Proof. By eqn. 32 and Facts 3.3, 3.4, and 3.5,

${6\;{\alpha\left( {{\lambda\; r} + \frac{G_{3}}{\lambda\; t} + \frac{G_{2}}{t}} \right)}} \leq f_{1} \leq {\beta\;{t.}}$ Also, eqn. 27 implies

${f_{1}/t} \geq {\frac{\delta}{n^{1 - {({2/k})}}}.}$ Thus, eqn. 33 follows from Theorem 2.1.

This describes an exemplary implementation of pick-and-drop sampling in accordance with the present invention, which can be represented in exemplary code as follows:

Algorithm 1 P&D (M, r, t, λ) Generate independent identically distributed random variables, {I_(j)}_(j=1) ^(r) with uniform distribution on [t]. S₁ = m_(1,I) ₁ C₁ = d_(1,I) ₁ q₁ = 1. for i = 2 → r do if (C_(i−1) < max{λq_(i−1,)c_(i)}) then S_(i) = s_(i) , C_(i) = c_(i) , q_(i) = 1 else S_(i) = S_(i−1) , C_(i) = c_(i) + fs_(i,l) , q_(i) = q_(i−1) +1 end if end for Output (S_(r),C_(r))

Theorem 3.7. Denote f_(i) ^(k)>100Σ_(j≠i)f_(j) ^(k) as a heavy element. There exist a (constructive) algorithm that makes one pass over the stream and uses O(n^(1−2/k) log(n)) bits. The algorithm outputs a pair (i,{tilde over (f)}_(i)), such that with probability 1. If there exists a heavy element f_(i), then also with constant probability, the algorithm will output (i, {tilde over (f)}_(i)), such that (1−ε)f_(i)≦{tilde over (f)}_(i).

Proof. Define t as in eqn. 27. Without loss of generality, we can assume F₁ is divisible by t. Note that if t>F₁ or f₁≧0.1F₁, then it is possible to find a heavy element with O(n^(1−2/k)) bits by existing methods such as in Moses Charikar, Kevin Chen, and Martin Farach-Colton. Finding frequent items in data streams. In ICALP '02: Proceedings of the 29th International Colloquium on Automata, Languages and Programming, pages 693-703, London, UK, 2002. Springer-Verlag, which is incorporated herein by reference. Otherwise, a stream D defines a matrix M for which we compute O(n^(1−2/k)/εδ) independent pick-and-drop samples. Since we do not know the value of δ, we should repeat the experiment for all possible values of δ. Output the element with the maximum frequency. With constant probability, the output of the pick-and-drop sampling will include a (1−ε) approximation of the frequency f_(i). Thus, there will be no other f_(j) that can give a larger approximation and replace a heavy element. The total space will define geometric series that sums to O(n^(1−2/k) log(n)).

If we know F₁ ahead of time then we can compute the value of t for any possible δ and thus solve the problem in one pass. However, one can show that the well-known doubling technique (when we double our parameter t each time the size of the stream doubles) will work in our case and, thus, one pass is sufficient even without knowing F₁.

Previously, such as described in Vladimir Braverman and Rafail Ostrovsky. Recursive sketching for frequency moments. CoRR, abs/1011.2571, 2010, which is incorporated herein by reference, we developed a method of recursive sketches with the following property: Given an algorithm that finds a heavy element and uses memory μ(n), it is possible to solve the frequency moment problem in space O(μ(n)log^((c))(n)). In this previous work, we applied recursive sketches with the method of Charikar et.al. cited above. Thus, we can replace the method from Charikar et al. with Theorem 3.7 and obtain:

Theorem 3.8. Let ε and k be constants. There exists a (constructive) algorithm that computes (1±ε)-approximation of F_(k), uses O(n^(1−2/5) log(n)log^((c))(n)) memory bits, makes one pass, and errs with probability at most ⅓.

The above analysis focuses on the case when 1 is a “heavy element,” but it is possible to repeat arguments for any i. The above Theorem 2.1 claims that 1 will be outputted with probability

$\Omega\left( \frac{f_{1}}{t} \right)$ for sufficiently large f₁. Notably, Theorem 2.1 holds for arbitrary distributions of frequencies. Furthermore, as addressed with respect to Theorem 3.6, there exist r,t,λ such that a bound similar to eqn. 3 holds. This new method can be combined with other method to obtain, for example, Theorem 3.8

Thus, the pick-and-drop sampling method samples a heavy element, such as an element i with frequency Ω(F_(k))) with probability Ω(1−/n^(1−2/k)) and gives approximation {tilde over (f)}_(i)≧(1−ε)f_(i). In addition, the estimations never exceed the real values, that is f_(i)≦f_(j) for all j. As a result, the space complexity of finding a heavy element can be reduced to O(n^(1−2/k) log(n)) bits. Recursive sketches can be used to resolve the problem with O(n^(1−2/k) log(n)log^((c))(n)) bits. Advantageously, optimizing the space complexity as a function of ε can be avoided.

The present invention has been described in terms of one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention. 

The invention claimed is:
 1. A database system comprising: a database; a database server configured to control reading data from and writing data to the database; an input to the database server configured to deliver a data stream formed of a sequence of elements, D={p₁, p₂, . . . , p_(m)} of size m of numbers from {1, . . . , n} to the database server; a non-transitive, computer-readable storage medium, having stored thereon, a computer program that, when executed by a processor, causes the processor to approximate frequency moments (F_(k)) in the data stream, such that a frequency of an element (i) is defined as f_(i)=|{j:p_(j)=i}| and a k-th frequency moment of D is defined as $F_{k} = {\sum\limits_{i = 1}^{n}m_{i}^{k}}$  in a single pass through the data stream by the steps of: (a) arranging a portion of the data stream in a matrix; (b) selecting an initial element in the matrix; (c) checking the matrix for a duplicate of the initial element; (d) upon identifying a duplicate of the initial element in the matrix, assuming that the initial element appears in each row of the matrix, assigning binary values to all other frequencies, and disregarding the initial element; (e) upon completing step (c) without identifying a duplicate of the initial element, assigning a binary value to all frequencies; (f) repeating steps (b) through (e) for a each subsequent element in the matrix; and (g) generating a report of approximated frequency moments in the data stream.
 2. The database system of claim 1 wherein the processor is further caused to implement a local counter to count a number of times an element appears in a suffix of a row in the matrix.
 3. The database system of claim 2 wherein the processor is further caused to implement a global counter incremented as a function of the local counter.
 4. The database system of claim 3 wherein the processor is further caused to drop and re-initiate the global counter if the local counter exceeds the global counter.
 5. The database system of claim 1 wherein the processor is further caused to approximate f_(i) as ≧ a fraction of f_(i) to limit memory resources used by the processor to estimate F_(k) to O(n^(1−2/k) log(n)) bits.
 6. A method for approximating frequency moments (Fk) in data streams, the method comprising the steps of: (a) arranging a portion of the data stream in a matrix, the data streams formed of a sequence of elements, D ={p₁, p₂, . . . , p_(m)} of size m of numbers from {1, . . . , n} such that a frequency of an element (i) is defined as f_(i)=|{j:P_(j)=i}| and a k-th frequency moment of D is defined as ${{f(k)} = {\sum\limits_{i = 1}^{n}m_{i}^{k}}};$ (b) selecting an initial element in the matrix; (c) checking the matrix for a duplicate of the initial element; (d) upon identifying a duplicate of the initial element in the matrix, assuming that the initial element appears in each row of the matrix, assigning binary values to all other frequencies, and disregarding the initial element; (e) upon completing step (c) without identifying a duplicate of the initial element, assigning a binary value to all frequencies; (f) repeating steps (b) through (e) for each subsequent element in the matrix; and (g) generating a report of approximated heavy elements in the data stream.
 7. The method of claim 6 wherein further comprising implementing a local counter to count a number of times an element appears in a suffix of a row in the matrix.
 8. The method of claim 7 further comprising implementing a global counter incremented as a function of the local counter.
 9. The method of claim 8 further comprising dropping and re-initiating the global counter if the local counter exceeds the global counter.
 10. The method of claim 6 further comprising approximating f_(i) as ≧ a fraction of f_(i) to limit memory resources used by the processor to estimate F_(k) to O(n^(1−2/k) log(n)) bits.
 11. The method of claim 6 further comprising limiting a degree of frequency moment (k) to greater than
 2. 12. A non-transitive, computer-readable storage medium, comprising: a computer program stored on the non-transitive, computer-readable storage medium that, when executed by a processor, causes the processor to approximate frequency moments (F_(k)) in the data stream, such that a frequency of an element (i) is defined as f_(i)=|{j:P_(j)=i}| and a k-th frequency moment of D is defined as ${f(k)} = {\sum\limits_{i = 1}^{n}m_{i}^{k}}$  in a single pass through the data stream by: (a) arranging a portion of the data stream in a matrix; (b) selecting an initial element in the matrix; (c) checking the matrix for a duplicate of the initial element; (d) upon identifying a duplicate of the initial element in the matrix, assuming that the initial element appears in each row of the matrix, assigning binary values to all other frequencies, and disregarding the initial element; (e) upon completing step (c) without identifying a duplicate of the initial element, assigning a binary value to all frequencies; (f) repeating steps (b) through (e) for a each subsequent element in the matrix: and (g) generating a report of approximated frequency moments in the data stream.
 13. The non-transitive, computer-readable storage medium of claim 12 wherein the processor is further caused to limit a degree of frequency moment (k) to greater than
 2. 